Skip to content

feat: enhance benchmark reporting by filtering out specified columns#742

Merged
vbreuss merged 1 commit intomainfrom
topic/enhance-benchmark-reporting-by-filtering-out-specified-columns
May 1, 2026
Merged

feat: enhance benchmark reporting by filtering out specified columns#742
vbreuss merged 1 commit intomainfrom
topic/enhance-benchmark-reporting-by-filtering-out-specified-columns

Conversation

@vbreuss
Copy link
Copy Markdown
Member

@vbreuss vbreuss commented May 1, 2026 •
edited
Loading

Updates the Nuke benchmark-comment generation to drop selected BenchmarkDotNet markdown table columns before posting results to PRs, and expands the PR CI benchmark matrix to run the new mock-creation benchmark suite.

Changes:

  • Filter out specific BenchmarkDotNet table columns (e.g., RatioSD, Gen0, Gen1) when generating the PR benchmark comment body.
  • Add MockCreationBenchmarks to the benchmark matrix in the PR CI workflow.

@vbreuss vbreuss self-assigned this May 1, 2026
@vbreuss vbreuss added the enhancement New feature or request label May 1, 2026
Copilot AI review requested due to automatic review settings May 1, 2026 12:17
Copilot AI reviewed May 1, 2026
View reviewed changes
Copy link
Copy Markdown

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Updates the Nuke benchmark-comment generation to drop selected BenchmarkDotNet markdown table columns before posting results to PRs, and expands the PR CI benchmark matrix to run the new mock-creation benchmark suite.

Changes:

  • Filter out specific BenchmarkDotNet table columns (e.g., RatioSD, Gen0, Gen1) when generating the PR benchmark comment body.
  • Add MockCreationBenchmarks to the benchmark matrix in the PR CI workflow.

Reviewed changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated 1 comment.

File Description
Pipeline/Build.Benchmarks.cs Filters benchmark markdown table output when composing the PR comment body.
.github/workflows/ci.yml Runs an additional benchmark suite (MockCreationBenchmarks) in PR CI.

Comment thread .github/workflows/ci.yml
@vbreuss vbreuss force-pushed the topic/enhance-benchmark-reporting-by-filtering-out-specified-columns branch from 4544cd9 to b1ead4b Compare May 1, 2026 12:24
@vbreuss vbreuss merged commit 99c558f into main May 1, 2026
13 checks passed
@vbreuss vbreuss deleted the topic/enhance-benchmark-reporting-by-filtering-out-specified-columns branch May 1, 2026 12:24
@github-actions
Copy link
Copy Markdown

github-actions Bot commented May 1, 2026 •
edited
Loading

🚀 Benchmark Results

Details 
BenchmarkDotNet v0.15.8, Linux Ubuntu 24.04.4 LTS (Noble Numbat)

Intel Xeon Platinum 8370C CPU 2.80GHz (Max: 3.37GHz), 1 CPU, 4 logical and 2 physical cores

.NET SDK 10.0.203

[Host] : .NET 10.0.7 (10.0.7, 10.0.726.21808), X64 RyuJIT x86-64-v4


Job=InProcess  Toolchain=InProcessEmitToolchain  IterationCount=15

LaunchCount=1  WarmupCount=10
Method Mean Error StdDev Ratio Gen2 Allocated Alloc Ratio
Mockolate 198.33 ns 6.180 ns 5.780 ns 1.00 - 1048 B 1.00
Imposter 302.46 ns 4.650 ns 4.350 ns 1.53 - 2248 B 2.15
TUnit.Mocks 42.81 ns 0.765 ns 0.716 ns 0.22 - 224 B 0.21
Moq 1,354.36 ns 9.059 ns 8.474 ns 6.83 - 2096 B 2.00
NSubstitute 2,051.74 ns 22.614 ns 21.153 ns 10.35 - 5048 B 4.82
FakeItEasy 1,729.37 ns 23.177 ns 21.680 ns 8.73 0.0019 2772 B 2.65
Details 
BenchmarkDotNet v0.15.8, Linux Ubuntu 24.04.4 LTS (Noble Numbat)

AMD EPYC 7763 2.74GHz, 1 CPU, 4 logical and 2 physical cores

.NET SDK 10.0.203

[Host] : .NET 10.0.7 (10.0.7, 10.0.726.21808), X64 RyuJIT x86-64-v3


Job=InProcess  Toolchain=InProcessEmitToolchain  IterationCount=15

LaunchCount=1  WarmupCount=10
Method Mean Error StdDev Ratio Allocated Alloc Ratio
Event_Mockolate 307.7 ns 6.12 ns 5.72 ns 1.00 1.83 KB 1.00
Event_Moq 16,025.0 ns 119.31 ns 99.63 ns 52.09 12.51 KB 6.84
Event_NSubstitute 5,931.3 ns 74.29 ns 69.49 ns 19.28 9.05 KB 4.95
Event_FakeItEasy 214,810.7 ns 1,744.82 ns 1,632.10 ns 698.26 15.39 KB 8.42
Event_Imposter 1,391.9 ns 34.01 ns 31.82 ns 4.52 8.8 KB 4.82
Event_TUnitMocks 192.1 ns 4.41 ns 4.13 ns 0.62 1.37 KB 0.75
Details 
BenchmarkDotNet v0.15.8, Linux Ubuntu 24.04.4 LTS (Noble Numbat)

AMD EPYC 7763 2.80GHz, 1 CPU, 4 logical and 2 physical cores

.NET SDK 10.0.203

[Host] : .NET 10.0.7 (10.0.7, 10.0.726.21808), X64 RyuJIT x86-64-v3


Job=InProcess  Toolchain=InProcessEmitToolchain  IterationCount=15

LaunchCount=1  WarmupCount=10
Method N Mean Error StdDev Ratio Allocated Alloc Ratio
Indexer_Mockolate 1 1,049.4 ns 61.20 ns 57.24 ns 1.00 3.9 KB 1.00
Indexer_Moq 1 222,563.2 ns 4,134.88 ns 3,867.77 ns 212.67 20.37 KB 5.23
Indexer_NSubstitute 1 9,679.4 ns 219.88 ns 205.67 ns 9.25 12.84 KB 3.29
Indexer_FakeItEasy 1 11,914.7 ns 198.17 ns 185.37 ns 11.39 13.88 KB 3.56
Indexer_Imposter 1 945.9 ns 69.42 ns 61.54 ns 0.90 5.16 KB 1.32

|
| Indexer_Mockolate | 10 | 2,594.2 ns | 36.96 ns | 30.87 ns | 1.00 | 4.95 KB | 1.00 |
| Indexer_Moq | 10 | 230,747.4 ns | 1,498.77 ns | 1,401.95 ns | 88.96 | 29.89 KB | 6.04 |
| Indexer_NSubstitute | 10 | 23,381.1 ns | 286.62 ns | 268.10 ns | 9.01 | 25.63 KB | 5.18 |
| Indexer_FakeItEasy | 10 | 25,455.1 ns | 298.77 ns | 264.85 ns | 9.81 | 35.47 KB | 7.16 |
| Indexer_Imposter | 10 | 2,196.7 ns | 47.32 ns | 44.26 ns | 0.85 | 7.97 KB | 1.61 |

Details 
BenchmarkDotNet v0.15.8, Linux Ubuntu 24.04.4 LTS (Noble Numbat)

AMD EPYC 7763 2.45GHz, 1 CPU, 4 logical and 2 physical cores

.NET SDK 10.0.203

[Host] : .NET 10.0.7 (10.0.7, 10.0.726.21808), X64 RyuJIT x86-64-v3


Job=InProcess  Toolchain=InProcessEmitToolchain  IterationCount=15

LaunchCount=1  WarmupCount=10
Method N Mean Error StdDev Ratio Allocated Alloc Ratio
Method_Mockolate 1 427.3 ns 7.39 ns 6.92 ns 1.00 2.15 KB 1.00
Method_Moq 1 183,986.2 ns 1,312.10 ns 1,227.34 ns 430.73 14.58 KB 6.78
Method_NSubstitute 1 5,944.8 ns 35.25 ns 32.97 ns 13.92 9.12 KB 4.24
Method_FakeItEasy 1 6,113.7 ns 39.94 ns 37.36 ns 14.31 8.11 KB 3.78
Method_Imposter 1 613.9 ns 8.17 ns 7.65 ns 1.44 4.04 KB 1.88
Method_TUnitMocks 1 653.5 ns 14.32 ns 13.39 ns 1.53 2.9 KB 1.35

|
| Method_Mockolate | 10 | 742.6 ns | 5.96 ns | 5.29 ns | 1.00 | 2.36 KB | 1.00 |
| Method_Moq | 10 | 187,331.1 ns | 1,217.39 ns | 1,138.75 ns | 252.28 | 18.48 KB | 7.83 |
| Method_NSubstitute | 10 | 8,825.8 ns | 80.89 ns | 75.66 ns | 11.89 | 12.07 KB | 5.12 |
| Method_FakeItEasy | 10 | 9,390.9 ns | 92.78 ns | 86.78 ns | 12.65 | 16.05 KB | 6.80 |
| Method_Imposter | 10 | 1,365.2 ns | 43.88 ns | 41.05 ns | 1.84 | 5.52 KB | 2.34 |
| Method_TUnitMocks | 10 | 1,706.5 ns | 30.09 ns | 28.15 ns | 2.30 | 4.49 KB | 1.90 |

Details 
BenchmarkDotNet v0.15.8, Linux Ubuntu 24.04.4 LTS (Noble Numbat)

AMD EPYC 7763 2.45GHz, 1 CPU, 4 logical and 2 physical cores

.NET SDK 10.0.203

[Host] : .NET 10.0.7 (10.0.7, 10.0.726.21808), X64 RyuJIT x86-64-v3


Job=InProcess  Toolchain=InProcessEmitToolchain  IterationCount=15

LaunchCount=1  WarmupCount=10
Method N Mean Error StdDev Ratio Allocated Alloc Ratio
Property_Mockolate 1 491.1 ns 2.30 ns 1.92 ns 1.00 2.46 KB 1.00
Property_Moq 1 11,745.0 ns 36.42 ns 32.29 ns 23.92 10.39 KB 4.22
Property_NSubstitute 1 7,119.7 ns 8.34 ns 7.40 ns 14.50 11.45 KB 4.65
Property_FakeItEasy 1 8,081.3 ns 31.91 ns 29.85 ns 16.46 11.24 KB 4.57
Property_Imposter 1 436.9 ns 1.10 ns 0.97 ns 0.89 3.13 KB 1.27
Property_TUnitMocks 1 768.6 ns 2.34 ns 2.07 ns 1.56 2.51 KB 1.02

|
| Property_Mockolate | 10 | 993.8 ns | 0.74 ns | 0.57 ns | 1.00 | 2.95 KB | 1.00 |
| Property_Moq | 10 | 18,026.1 ns | 60.92 ns | 54.01 ns | 18.14 | 18.28 KB | 6.19 |
| Property_NSubstitute | 10 | 16,509.6 ns | 24.06 ns | 22.50 ns | 16.61 | 21.08 KB | 7.14 |
| Property_FakeItEasy | 10 | 19,455.6 ns | 39.94 ns | 33.35 ns | 19.58 | 30.81 KB | 10.43 |
| Property_Imposter | 10 | 1,106.3 ns | 2.91 ns | 2.58 ns | 1.11 | 4.67 KB | 1.58 |
| Property_TUnitMocks | 10 | 2,198.5 ns | 8.20 ns | 6.85 ns | 2.21 | 4.66 KB | 1.58 |

Details 
BenchmarkDotNet v0.15.8, Linux Ubuntu 24.04.4 LTS (Noble Numbat)

AMD EPYC 9V74 2.60GHz, 1 CPU, 4 logical and 2 physical cores

.NET SDK 10.0.203

[Host] : .NET 10.0.7 (10.0.7, 10.0.726.21808), X64 RyuJIT x86-64-v3


Job=InProcess  Toolchain=InProcessEmitToolchain  IterationCount=15

LaunchCount=1  WarmupCount=10
Method Mean Error StdDev Ratio Allocated Alloc Ratio
Callback_Mockolate 362.2 ns 1.91 ns 1.69 ns 1.00 1.79 KB 1.00
Callback_Moq 71,998.8 ns 760.48 ns 674.14 ns 198.79 8.88 KB 4.97
Callback_NSubstitute 4,245.7 ns 102.25 ns 90.64 ns 11.72 7.74 KB 4.33
Callback_FakeItEasy 4,349.4 ns 79.52 ns 74.38 ns 12.01 6.81 KB 3.80
Callback_Imposter 400.2 ns 2.18 ns 1.93 ns 1.11 2.38 KB 1.33
Callback_TUnitMocks 651.9 ns 7.52 ns 7.04 ns 1.80 2.63 KB 1.47

@sonarqubecloud
Copy link
Copy Markdown

sonarqubecloud Bot commented May 1, 2026

Quality Gate Passed Quality Gate passed

Issues
0 New issues
0 Accepted issues

Measures
0 Security Hotspots
0.0% Coverage on New Code
0.0% Duplication on New Code

See analysis details on SonarQube Cloud

@github-actions
Copy link
Copy Markdown

github-actions Bot commented May 2, 2026

This is addressed in release v3.1.0.

@github-actions github-actions Bot added the state: released The issue is released label May 2, 2026
This was referenced May 2, 2026
Merged
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

Copilot code review Copilot Copilot left review comments

Assignees

@vbreuss vbreuss

Labels

enhancement New feature or request state: released The issue is released

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@vbreuss